Exemplary embodiments of the present invention relate to a method for predicting an interface control action of a user with an in-vehicle user interface. Exemplary embodiments of the invention further relate to an in-vehicle user interface with a memory unit, a processing unit, and a display unit.
Many vehicles currently have an in-vehicle user interface providing a user with a vast amount of possible interface control actions. For example, a single in-vehicle user interface and/or different applications inside the vehicle may be used to choose a variety of actions such as listen to music, make a phone call, and adjust the temperature in the vehicle. All these different interface control actions usually cannot be presented to a user at the same time. This results in complicated menus, where the user has to perform several tasks of choosing sub-menus and options, before an interface control action finally is selected. Several efforts have been met already to increase the usability of user interfaces.
The European patent document EP 2 369 299 A1 and US patent document US 2013/0166096 A1 both disclose a predictive destination entry system for a vehicle navigation system to aid in obtaining destinations for a vehicle. The navigation system comprises a memory for storing data, a display coupled with the vehicle, and a processor coupled with the display and with a memory. The processor is configured to predict a destination for the vehicle based on the data stored in the memory.
US patent document U.S. Pat. No. 8,429,103 B1 discloses an apparatus and methods for providing machine-learning services for a mobile platform. A machine-learning service executing on a mobile platform can receive data related to a plurality of features, wherein the received data comprises a called party of a telephone call to be originated by the mobile platform. The output by the machine-learning services comprises a prediction of a volume setting and/or a mute setting of the mobile platform.
Exemplary embodiments of the invention are directed to improving the usability of an in-vehicle user interface.
Exemplary embodiments of the invention improve the usability of an in-vehicle user interface using a method for predicting an interface control action of a user of such an in-vehicle user interface with several steps. A first step comprises collecting and storing data, at least vehicle data about the vehicle and its environment from at least one sensor of the vehicle and user data about user interactions with the user interface and/or different applications inside the vehicle. This collecting and storing of data can be performed prior to the consequent steps, but also prior and/or in parallel to the consequent steps, meaning that data can be collected and stored as a background process. The second step comprises assigning likelihoods to at least two possible interface control actions by the user based on the collected and stored data. This assigning of likelihoods can be a continuous process, where the likelihoods are updated in specific time-intervals or whenever new data is collected and/or stored. The third step comprises determining at least one most likely interface control action from the likelihoods. In particular, the at least one most likely interface control action may be a subset of any size of the complete set of possible interface control actions. The fourth step comprises providing the user with the at least one most likely control action so that it is selectable and performable with one single user interaction with the user interface. All the above steps can be performed consecutively or in parallel, meaning that data, likelihoods, and the at least one most likely interface control action may be dynamic. So, they can be updated after a fixed time-interval or whenever new data is available.
The resulting advantage is that only one user interaction with the user interface is necessary for a desired control action or task. Furthermore, only a small set of interface control actions is shown to the user at a time so the user can perform an interface control action fast and is not distracted by a huge amount of possible interface control actions at a time. As the likelihoods that determine which interface control actions are made available to the user depend on the data, that is, prior information about the vehicle, its environment and user interactions with the user interface and/or different applications inside the vehicle, the method provides the user with situational or context-specific control actions. Accordingly, it is possible that in a specific situation or context, interface control actions that, based on collected and stored data, are most likely for the user choose are made available to the user. Hence the usability of an in-vehicle user interface is increased in a dynamic way.
The method can be improved, so that a larger likelihood is assigned to frequently used interface control actions when assigning the likelihoods. The resulting advantage is that it is more likely that interface control actions that are chosen more frequently are made available to the user, which means they are easier and faster to choose.
The method can be further refined so that a larger likelihood is assigned to recently used interface control actions when assigning the likelihoods. This results in a more user friendly method, since, if the user picks up a new habit, such as using a previously unused interface control action or using a previously used interface control action in a new context, a larger likelihood is assigned to this new habit, i.e., the respective interface control action. As a result, the respective interface control action is made available to the user in the “making available” step much easier than in one of the previously described embodiments of the invention. Accordingly, the method quickly adapts to changing habits of the user, as the adaptation is not, or less, influenced by old patterns of interface control actions of the user.
A further refinement of the method is that only data younger than a given age are taken into account when assigning the likelihoods. This ensures that very old data, data that is not relevant for the prediction of currently preferred interface control actions of the user, does not influence the outcome of the method. Furthermore, computing time and memory is saved.
In addition, it is possible that collecting and storing data includes collecting and storing data, in particular data about the user, from and to a cloud. Here, the term “cloud” refers to a remote IT infrastructure. That means pre-processed data as well as events or further data from the outside world, which are not available to sensors of the car or the user interface directly, can be used to predict an interface control action.
Next, the methods can be improved if a classifier is built to calculate the likelihoods for the at least two possible interface control actions based on the collected and stored data, and these calculated likelihoods are then assigned to the at least two possible interface control actions. The use of a classifier gives the advantage that a systematic approach is realized for achieving a local, dynamic learning system that takes advantage of a vast amount of already existing knowledge to increase the usability of the user interface. Accordingly, it is possible to choose a suitable classifier out of the many classifiers that already exist.
When choosing the respective classifier for the embodiment of the invention, one only has to remind several criteria or needs that have to be met by a suitable classifier. First, the classifier needs to be an online or iterative learning algorithm. This means training can be done dynamically, that is, by taking one data point at a time instead of requiring a lot of saved data before any training can take place. Second, the classifier needs to work well with a so-called “partial-state”. That means if some data is not available, i.e. a data set is not complete, the classifier should still give reasonable likelihoods using whatever data is available. This is especially important when data from the cloud is used, as the cloud is not always available or reliable. Third, the classifier should not need to know the total number of classes, which corresponds to the number of control actions, beforehand, so that the classifier grows as it is trained by more and more control actions that are discovered and used by the user. Fourth, the classifier needs to be simple to implement, easy to debug, and robust. Fifth, it would be advantageous to construct an ensemble version of the classifier with a single classifier model without data redundancy. This allows use of an ensemble classifier in which variance and bias trade-off can be fine-tuned in real time by changing ensemble hyper parameters. One classifier that meets these requirements is the naïve Bayes classifier. But in principle any classifier meeting the above criteria is fit for the inventive method.
It is also possible to combine different classifiers so as to compensate for specific disadvantages of a given classifier. In particular, a naïve Bayes classifier can be used as a primary learner for the method so as to get a first estimate of the likelihoods; then e.g. a random forest classifier can be used as a supplement to increase the overall accuracy of the method under specific circumstances. When those circumstances are not met, the method simply relies then on the results of the naïve Bayes classifier.
The method can further be modified so that the classifier is an ensemble classifier with several individual classifiers, where each individual classifier is built on a data chunk of the collected and stored data. This provides the advantage of a very flexible classifier that can adapt to a growing set of collected and stored data, in particular by building a new individual classifier for newly collected and stored data. Furthermore, specific sets or sub-sets of data can be ignored by erasing the corresponding individual classifier that is built on the data to be left out.
A further refinement of the method can be achieved if the data chunks are weighted with a bias depending on the age of the collected and stored data contained within, in particular with the bias giving more weight to data chunks with recent data. Equivalently, the individual classifiers built on the respective data chunks can be weighted with a bias. This adds a selection bias to our ensemble classifier, more specific, it adds a temporal bias that takes into account recently collected data more than older data.
This gives two technical advantages. First, our specific realization of the bias allows the assumption of a static setting, since the data in each chunk is static and unchanging. Thus, algorithms and techniques that excel in static situations can be used for the classifiers. For example, both the Bayesian and random forest classifiers described above assume static correlations. The combination of such “static” classifiers with the weights, however, results in the ensemble classifier approximating a dynamic system, as the weights are dynamic. Since people change, user behavior is typically dynamic, and, thus, temporal bias is critical in predicting interface control action of a user. Second, the temporal bias provides a memory horizon, where data older than a certain amount or age can no longer influence the performance of the ensemble classifier. Thus, the long-term resource use of the user interface can be tweaked by a clever manipulation of this memory horizon, that is, by choosing a suitable weight-decay for older data and by not storing, that is, by deleting data stemming from before the memory horizon and the respective classifiers. This is much cheaper than storing all data for the lifetime of the vehicle or user.
As for modeling the weight-decay, different schemes, in particular a linear or an exponential scheme can be used. In a linear scheme the weights decay by a fixed amount. In case of an exponential decay, the weights would decay by an amount relative to each other.
With regard to predicting user behavior, it is useful to have a higher weight-decay for more dynamic users, since older data is not useful for a very dynamic user. Also it is useful to have a lower weight-decay for less dynamic users, where old data is still useful for identifying correlations in user and vehicle data. However, for less dynamic users, recent data and older data should share very similar correlations, and thus, it is not as critical to keep around old data for those less dynamic users either. A high weight-decay is thus desirable in both cases. The trade-off here is accuracy. A classifier needs a certain critical mass of data based on the complexity of the classifier being used. Thus, the key is to pick the highest weight-decay possible that still keeps the desired critical mass of data based on the chosen algorithm for the classifier. This is very much related to the bias/variance trade-off frequently spoken about in machine learning literature. Specifically, a high weight-decay does well with high bias and low variance algorithms or classifiers.
The weighting results in the overall advantage that classifiers that are based on new data, which means data stemming from recent behavior or situations, have more influence on the output of the ensemble classifier, that is, on the likelihoods assigned to the interface control actions. As a consequence, the method adapts better to changing habits of a user.
Finally, it is advantageous if each data chunk contains all data, or data points, stemming from a time-interval specific for each data chunk, where, in particular, time-intervals assigned to one data chunk may overlap with time-intervals assigned to other data chunks, so that specific data may be part of more than one data chunk. In particular, there are two parameters related to chunk overlap: overlap percentage and stack depth. Overlap percentage refers to which percentage data samples exist in multiple data chunks. For example, a 10% overlap means, that, if sampled randomly, only 10% of the samples' data points would be present in multiple data chunks. 100% overlap would mean that every sample is present in multiple data chunks. Coupled with data chunk width, this parameter controls when a new data chunk must be created. Stack depth refers to the maximum number of data chunks a single data sample can be present within. For example, a stack depth of two means that a data sample or data point can exist in at most two different data chunks. Accordingly, combining stack depth with chunk overlap, a data chunk overlap of 100% with a stack depth of two would mean that every sample is present in exactly two data chunks.
This results in the advantage that each individual classifier, corresponding to the data chunk it is built on, represents correlations in the data in a specific time slot. Accordingly, specific time-intervals, such as, e.g., time-intervals during holidays, can be easily selected, and consequently ignored or favored for predicting user behavior. By setting the length of the time-intervals, the quality of each individual classifier can be adjusted. Here, a long time-interval leads to an enhanced accuracy of the individual classifiers in combination with a reduced flexibility of the ensemble classifier, and a short time-interval leads to a reduced accuracy of the individual classifiers in combination with a very flexible ensemble classifier. Furthermore, the more the time-intervals of distinct data chunks overlap the smoother the behavior of the ensemble classifier gets, preventing “jumps” in the predicting of user behavior that might surprise and irritate the user.
It is also advantageous to choose a stochastic overlap percentage, in particular with a mean of 100%. This means that, on average, the chunk overlap percentage is, e.g., 100%. In this case, the random nature of this setting helps to control the influence of the bias caused by the selection of the data chunks on the performance on the ensemble classifier. With the random nature, the desired temporal bias is still present, but the undesired bias related directly to the data chunk width is mitigated. It is advantageous to maintain a stack depth of exactly two. Ideally, these parameters are fine-tuned to each particular user in order to achieve the best prediction results of the ensemble classifier.
The invention further provides an in-vehicle user interface with a memory unit, a processing unit and a display unit. Here, the memory unit is for collecting and storing data, at least vehicle data about the vehicle and its environment from at least one sensor of the vehicle and user data about user interactions with the user interface and/or different applications inside the vehicle, the processing unit, with access to the memory unit, is for assigning likelihoods to at least two possible interface control actions based on the collected and stored data, and for determining at least one most likely interface control action from the likelihoods, and the display unit is for making available, in particular presenting, to the user the at least one most likely control action so that it is selectable and performable with one single user interaction with the user interface.